Rigorizing and Extending the Cox–Jaynes Derivation of Probability: Implications for Statistical Practice

There have been three attempts to date to establish foundations for the discipline of probability, namely the efforts of Kolmogorov (who rigorized the frequentist approach), de Finetti (who gave Bayesian notions of belief and betting odds a formal treatment) and RT Cox/ET Jaynes (who developed a Bayesian theory of probability based on reasonable expectation (Cox) and the optimal processing of information (Jaynes)). The original “proof” of the validity of the Cox–Jaynes approach has been shown to be incomplete, and attempts to date to remedy this situation are themselves not entirely satisfactory. Here we offer a new axiomatization that both rigorizes the Cox–Jaynes derivation of probability and extends it — from apparent dependence on finite additivity to (1) countable additivity and (2) the ability to simultaneously make uncountably many probability assertions in a logically-internally-consistent manner — and we discuss the implications of this work for statistical methodology and applications. This topic is sharply relevant for statistical practice, because continuous expression of uncertainty — for example, taking the set Θ of possible values of an unknown θ to be (0, 1), or R, or the space of all cumulative distribution functions on R — is ubiquitous, but has not previously been rigorously supported under at least one popular Bayesian axiomatization of probability. The most important area of statistical methodology that our work has now justified from a Cox–Jaynes perspective is Bayesian non-parametric (BNP) inference, a topic of fundamental importance in applied statistics. We present two interesting foundational findings: (1) Kolmogorov’s probability function PK(A) of the single argument A is isomorphic to a version of the Cox–Jaynes two-argument probability map PCJ(A | B) in which Kolmogorov’s B has been hard-wired to coincide with his sample space Ω, and (2) most or all previous BNP work has actually been foundationally supported by a hybrid frequentist-Bayesian version of Kolmogorov’s probability function in which parameters are treated as random variables (an unacceptable move from the frequentist perspective); this previous BNP work is methodologically sound but is based on an awkward blend of frequentist and Bayesian ideas (whereas our Cox–Jaynes BNP is purely Bayesian, which has interpretational advantages).